fix(agents): non-streaming fallback for Ollama agent chat#343
Conversation
There was a problem hiding this comment.
Sorry @w7-mgfcode, you have reached your weekly rate limit of 500000 diff characters.
Please try again later or upgrade to continue using Sourcery
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Root cause
Interactive Agent Chat (WebSocket
/agents/stream→AgentService.stream_chat→agent.run_stream) fails on a local-Ollama agent: Ollama's OpenAI-compatible endpoint rejects PydanticAI's streamed request with400 - {'message': 'invalid message content type: <nil>'}for both the primary (ollama:qwen3:8b) and fallback (ollama:llama3.1:8b), soFallbackModelraises aFallbackExceptionGroupand the UI shows "Stream error: All models from FallbackModel failed."The non-streaming path (
agent.run()inchat()) works fine on Ollama (the showcaseagent_hitl_flowuses it). Cloud providers stream fine. So it's an Ollama OpenAI-compat + streaming incompatibility (a streamed-request message is serialized withcontent: null).Fix
stream_chatnow detects the provider fromagent_default_model. For theollamaprovider it uses the non-streamingagent.run()and emits the reply as a singletext_delta+ the existingapproval_required(#336) +completeevents. Cloud providers keep the truerun_streamtoken-streaming path unchanged. The post-run logic (session persistence,deps.pending_action→awaiting_approval→approval_required, completion) is now shared by both paths.Tests
test_stream_chat_ollama_uses_nonstreaming_path— withagent_default_model=ollama:*,stream_chatcallsagent.run()(assertsrun_streamis not called), emitstext_delta+approval_required(from a gateddeps.pending_action) +complete, and flips the session toawaiting_approval.test_stream_chat_cloud_keeps_streaming_path— with a cloud model,run_streamis used andrunis not (regression guard).agent_default_modelso they exercise therun_streampath deterministically regardless of the local.env(these were.env-coupled — they pass in CI where the default isanthropic:..., but the fix makes them robust).Validation
ruff check✅ ·ruff format --check✅mypy app/✅ — only pre-existingxgboostoptional-dep import errors in untouched filespyright app/features/agents✅ 0 errorspytest -m "not integration"✅ 1655 passed, 12 skippedNotes
Closes #342